The present invention relates generally to the field of machine learning, and more particularly to, systems and methods for improving the performance of systems and applications implementing machine learning.
Machine learning may be used in a variety of systems and applications for performing automated and semi-automated tasks such as problem solving, decision making and prediction. In the field of image processing, machine learning systems may be used for object or image recognition, for example, in applications such as face-recognition, hand-written digit recognition, and obstacle recognition in self-navigating vehicles. In the medical field, machine learning may be used for applications such as medical diagnostics or bioinformatics. In industrial fields, machine learning may be used for applications such as fault detection in networks and industrial systems.
Machine learning requires the development of a model using training data relating to a number of features or variables. Once a satisfactory model is developed, the model can be applied to real world data to provide an outcome for the relevant task. In most applications, large quantities of high quality training data are needed for good system performance, typically relating to large numbers of features.
As the number of features used in the model and training data increases, various problems arise. This is known as “the curse of dimensionality”. In particular, the higher the data dimensionality (i.e., the higher the number of features) the worse the performance degradation and the less predictable the behavior on real unseen data. In addition, as data dimensionality increases, the execution/training times increase together with the computational resources required by the system. Moreover, the higher the data dimensionality, the greater the lack of knowledge by administrators of the system about the most useful features/components in the system, and the greater the effects of feature inter-dependence in the system.